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Abstract — In this paper we propose an XML-based multi- 
agent recommender system for supporting online recruitment 
services. Our system is characterized by the following features: 
(i) it handles user profiles for personalizing the job search over 
the Internet; (ii) it is based on the Intelligent Agent Technology; 
(Hi) it uses XML for guaranteeing a light, versatile and standard 
mechanism for information representation, storing and exchange. 
The paper discusses the basic features of the proposed system, 
presents the results of an experimental study we have carried 
out for evaluating its performance, and makes a comparison 
between the proposed system and other e-recruitment systems 
already presented in the past. 

Index Tenns — Multi- Agent Systems, XML, E-Services, Re- 
cruitment Services. 



I. Introduction 

IN the last decade Internet services emerged as a significant, 
both social and cultural, phenomenon; in fact, presently, 
many organizations provide their customers with the possibil- 
ity to access their services also on the Internet. 

In this scenario, online recruitment services, supporting 
both individuals looking for a job and companies looking 
for employees, are assuming a prominent role. In such a 
context, generally, companies populate a database of job 
proposals and individuals are supported in their job search 
by an engine based on classical Information Retrieval (IR) 
technique^ Several studies show that the number of users 
accessing these services is dramatically increasing [1]; as a 
consequence, it is possible to foresee that, in the next years, a 
huge amount of job proposals will be handled by means of the 
Internet. In such a situation, classical IR techniques used by the 
present recruitment services could provide an individual with 
an excessively large number of job proposals not interesting 
for him. This could result in a low user perceived quality of 
service and, ultimately, in his decision to not access these 
services. 

A way for solving this problem consists of realizing person- 
alized search engines, which combine classical IR techniques 
and user modeling methodologies [2]. In the literature some 
approaches based on recommender systems [3] have been 
proposed to personalize search engines in various applica- 
tion contexts, such as Web browsing [4], e-commerce [5], 
e-learning [6], and so on. In addition, in some proposals, 
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recommender systems have been combined with the agent 
technology for making them autonomous, reactive and pro- 
active [7]. This paper provides a contribution in this setting; 
in fact, it proposes an XML-based multi-agent recommender 
system that exploits rich user profiles to support personalized 
recruitment services. 

Our system is based on the agent technology. In fact, a 
User Agent is associated with each user and manages his 
profile, as well as the interaction with him; a Recruitment 
Agent supports each User Agent in the selection of those 
job proposals appearing to be the most adequate for the 
corresponding user The exploitation of the intelligent agent 
technology allows our system to improve its autonomy and 
adaptiveness, as well as to easily partition the various tasks 
among several entities, thus improving its scalability. Finally, 
it allows an easy management of existing legacy systems by 
means of suitable wrappers; as a consequence, our system can 
easily interact with pre-existing recruitment Web sites and this 
contributes to enhance the completeness of obtained results. 

Our system is XML-based. XML is a standard language 
for representing and exchanging information. It embodies 
both representation capabilities, typical of HTML, and data 
management features, typical of DBMSs. XML information 
bases are plain text documents and, consequently, they are 
light, versatile, easy to be exchanged and can reside on 
different, both hardware and software, platforms. In spite of 
this simplicity, the information representation rules embodied 
in this language are powerful enough to allow the management 
of sophisticated queries. 

As for the contribution of our paper in the field of e- 
recruitment systems, we observe that: 

• In order to select the job proposals that best satisfy user 
requirements, a traditional e-recruitment system considers 
only the query of the user (that specifies his preferences 
and constraints), along with the features of available job 
proposals. On the contrary, our system takes into account 
not only the query of the user but also his reaction to 
previous job proposal recommendations it made to him. 
Moreover, it analyzes similitudes and/or correlations 
among job proposals to produce its suggestions; in this 
way, it can highlight some proposals that are usually 
ignored by traditional e-recruitment systems and that 
might be appealing for the user 

• In a traditional e-recruitment system, a user can query 
the corresponding (internal) database. As a consequence, 
if the system's answer to a query is insufficient, the 
user must access other e-recruitment systems and re- 
submit the query; after this, he must compose the results 
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provided by each system to construct his final list of 
job proposals. This activity might be time consuming, 
since the user is obliged to contact and query several 
e-recruitment systems. In addition, it might be prone 
to errors, since two e-recruitment systems might exploit 
different methodologies and use different terminologies 
to both select and represent job proposals. Specifically, 
as for methodologies, it could happen that the same job 
proposal for a user Ui receives contrasting scores by 
different e-recruitment systems. As a consequence, when 
Ui compares the results provided by the various systems, 
he might be confused and, therefore, incapable of making 
a decision about its suitability for him. As for termi- 
nologies, two independent e-recruitment systems might 
present to a user the same job proposal by exploiting 
different terms. As an example, a job proposal regarding 
the design, the development and the validation of software 
programs might be called "Program Application Support 
SpeciaUst" by an e-recruitment system, and "Software 
Test and Software Design Engineer" by another one. In 
situations like that previously described, since the two 
systems are totally independent each other, there is no 
form of coordination and, consequently, no way to inform 
a user that two or more terms represent the same job 
proposal. 

Our system is capable of facing all these problems; specif- 
ically, as it will be clarified in the following, it continuously 
monitors existing online recruitment services and collects job 
proposals coming from them; as a consequence, user search is 
not restricted to the database associated with an e-recruitment 
system but it spreads across disparate sources. This system 
organization allows a user to pose his queries in a transparent 
way; in other words, in order to obtain an answer, he could 
be unaware of the models and the languages of the involved 
databases, as well as of their internal structure. 

As for the Recommender Systems research area, the system 
we propose in this paper provides the following novelties: 

• A traditional Recommender System usually identifies the 
"Top K" recommendations, i.e., the K information items 
(in our case, job proposals) having the highest relevance 
for the user. The coefficient K is usually constant and 
pre-defined. In line with current advances [8], our system 
can analyze user reactions to past proposals to assess if 
the number of job proposals it presents to users is too 
low (resp., too high) and, in the affirmative case, it can 
dynamically augment (resp., reduce) this number 

• In content-based Recommender Systems, a user profile is 
generally represented as a vector of pairs {ki,Wi), where 
ki is a keyword and Wi is a weight denoting the impor- 
tance of ki for the user. On the contrary, in our system, 
user profiles are richer and take into account various 
characteristics and preferences of involved users. Even 
if this implies a higher complexity in their management, 
the improvement of results that can be obtained is very 
satisfying, as it will be clarified in Section IV-EI 

• Our approach applies in the Recommender Systems field 
some mathematic tools generally used in other appli- 




Fig. 1 . General architecture of our system 

cation contexts, such as manufacturing engineering [9], 
bio informatics [10] and econometrics [11]. Interestingly 
enough, the computational effort it requires is not partic- 
ularly heavy, as it will be demonstrated in Section lTV-B.ll 

II. General Architecture of our system 

Our system consists of two main agent typologies, namely: 
(i) a User Agent {UA), that manages the profile of a user, as 
well as the interaction with him, and ( ii) a Recruitment Agent 
(RA), that handles both job proposals and recommendation 
activities. 

Moreover, a Job Proposal Database (JPD) is used to 
store all available job proposals. JPD can be populated by 
companies who can insert, remove or modify job proposals by 
means of a Company Agent, i.e., a suitable Interface Agent, 
analogous to that described in [12]. In addition, JPD can be 
automatically enriched by suitable Wrapper Agents, analogous 
to that described in [13], that continuously monitor existing 
online recruitment services, find new job proposals and store 
them in JPD, if they are not already present. JPD can 
be described by an XML document; the corresponding XML 
Schema is shown in the Appendix available at the address 

http : / /www .ing.unirc. it /ursino/tsmcO 5 /Appendix . pdf 

Specifically, JPD stores a set of job proposals; 
a job proposal JPi is described by a tuple 
{JIDi, JURLi, JTopicSeti, JCharacteristicSeti), where: 

• JIDi is an identifier; 

• JURLi is the url where the complete description of the 
job proposal is available; 

• JTopicSeti — {JTopici-^, JTopici^, . . . , JTopici^} is 
a set of topics describing J Pi; a topic is characterized by 
its name; 

• JCharacteristicSeti is a dynamic set of characteristics 
associated with J Pi. Each characteristic is described by 
a pair {Feature, Value); some of the possible features 
are the salary associated with the job proposal, the 
city/country of the job proposal, the foreign languages, 
the skills, the years of experience and the academic title(s) 
required from the candidate. 

Observe that we have chosen to store available job proposals 
into an independent knowledge base and not in the ontology 
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of a specific agent; in this way, several independent and 
specialized agents can operate in our system to automatically 
and continuously update and refine them. 

The general architecture of our system is shown in Figure [T] 
Its behaviour is as follows. When a user m, wants to perform 
a job search, he submits a query to the associated User Agent 
UAi. After this, UAi contacts RA and provides it with both 
the query and the profile of Ui. RA selects from JPD the 
available job proposals satisfying user query, ranks them on the 
basis of the presumed user preferences and selects those ones 
that best fit the past interests of m, as well as other interests, 
someway related to those Ui considered in the past and that he 
still disregarded. After this, RA sends the selected proposals 
to UAi that presents them to uf, UAi monitors Ui and, when 
he performs his choices, it updates his profile accordingly. 

As previously pointed out, the role of XML in our system 
is crucial. In fact: 

• Agent ontologies are stored as XML documents; as a con- 
sequence, they are light, versatile, easy to be exchanged 
and can reside on different hardware and software plat- 
forms. In spite of this simplicity, the information repre- 
sentation rules embodied in XML are powerful enough 
to allow a sophisticated information management. 

• The agent communication language adopted in our sys- 
tem is ACML [14]; this is the XML encoding of the 
standard Agent Communication Language. 

• The extraction of information from the various data 
structures is carried out by means of XQuery [15]. 
This is becoming the standard query language for the 
XML environment. Since XQuery is based on the XML 
framework, it can handle a large variety of data. It has 
capabilities typical of database query languages as well 
as features typical of document management systems. 

• The manipulation of agent ontologies is performed by 
means of the Document Object Model (DOM) [16]. DOM 
makes it possible for programmers to write applications 
working properly on all browsers and servers as well 
as on a large variety of both hardware and software 
platforms. 

The architecture of our system can be considered mixed, i.e., 
partially centralized and partially distributed. In this sense it 
follows the ideas expressed in [17], where an architecture for 
handling telecommunication networks, based on the presence 
of an auctioneer agent that carries out most of the negotiation 
activities, is described. It can be considered centralized for 
the presence of the Recruitment Agent that performs most of 
the activities concerning the selection of the job proposals best 
fitting user exigencies. On the other hand, it can be assumed to 
be distributed for the presence of many User Agents, Wrapper 
Agents and Company Agents that continuously cooperate with 
the Recruitment Agent for performing the whole recruitment 
task. 

Observe that, in our system, there exists only a central 
Recruitment Agent. Such a choice is justified by the following 
motivations: 

• Network Congestion. In our architecture, each User Agent 
is in charge of forwarding both the User Profile and the 



user query to the Recruitment Agent that processes this 
query and sends the corresponding results to it; as a 
consequence, the two agents exchange a small number 
of (generally simple) messages and this prevents network 
overload. In a distributed implementation of the Recruit- 
ment Agent activities, various Recruitment Agents would 
cooperate for processing user queries. As a consequence, 
they should continuously communicate and exchange 
messages; this would generate a high network traffic 
and the whole system would quickly be overwhelmed 
with messages among agents. This problem is particularly 
relevant in our reference scenario where the number 
of job seekers, and consequently of User Agents, is 
very high; this would cause an overwhelming amount of 
exchanged messages and, ultimately, significant latencies 
and users dissatisfaction. 

« Task Assignment. In a distributed implementation of 
the Recruitment Agent activities, a query is propagated 
through several Recruitment Agents that spend memory 
and CPU resources to process it. In this scenario, it 
could happen that some Recruitment Agents would be 
overwhelmed with requests, whereas other ones would be 
almost unoccupied. In order to optimize the system per- 
formances, it would be necessary to define protocols for 
intelligently assigning tasks to the various Recruitment 
Agents. These protocols appear very complex to be real- 
ized because the potential job seekers (and, consequently, 
the potential User Agents) might be very numerous and 
geographically sparse. 

« Message Duplication. In a distributed implementation of 
the Recruitment Agent activities, multiple copies of a 
query might be received by the same Recruitment Agent 
(we call these copies duplicated messages). To better 
clarify this concept and its consequences, consider four 
Recruitment Agents RAd^, RAd2, RA^r^ and RAd^, and 
assume that: (i) a user submits a query Q to RAd^', (ii) 
RAdi is not capable of answering Q; (Hi) RAd^ asks 
RAd2 and RAd^ to answer Q; (iv) RAd2 and RAd;^ are 
only capable of producing a partial answer to Q (e.g., 
the number of job proposals retrieved by them is lesser 
than that required by the user); (v) RAd2 and RAd.^ might 
independently forward Q to RAd^ that would receive and, 
in its turn, could process and possibly transmit Q twice. 
This example clearly shows that duplicated messages are 
pure overhead; they increase the network congestion and 
determine a waste of CPU and memory resources for the 
Recruitment Agents that are often obliged to process the 
same query more than once. In spite of the resource waste 
caused by them, they do not improve system accuracy 
since they do not contribute to increase the chance of 
finding job proposals relevant for user. 
These problems are emphasized in an e-recruitment sce- 
nario, where the number of job seekers contemporarily 
accessing our system could be very high; in this case, the 
probability of duplicating messages might be extremely 
high. As a consequence, suitable mechanisms for iden- 
tifying duplicated messages and avoiding to process and 
forward them are required; however, these mechanisms 
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are generally difficult to be designed and implemented. 

Interestingly enough, these motivations are in accordance 
also with the ideas illustrated in [18]. 

Finally, we would like to point out that our system does 
not provide for any additional layer between the User Agent 
and the Recruitment Agent. This choice is justified by the 
following motivations: 

• Ability to cooperate with other e- recruitment systems. 
One of the most important capabilities of our system 
consists of its ability to effectively cooperate with other e- 
recruitment systems (see below). A layered architecture 
does not appear particularly suitable for pursuing such 
an objective. In fact, in a layered architecture, system 
functionalities are associated with different abstraction 
levels, each represented by a layer; the abstraction degree 
generally increases from the lowest levels to the highest 
ones. The presence of different abstraction levels has the 
following consequences: 

- The interaction between two layered systems re- 
quires each of them to have a deep knowledge of the 
architectural features of the other one. In fact, each 
system should know the abstraction level associated 
with each layer of the other system to properly ex- 
change information and cooperate. Such a knowledge 
is often missing and, hence, the mapping from a layer 
in a system to a layer in another one might be not 
obvious and, in some cases, not possible. 

- Many systems cannot be easily structured in a lay- 
ered fashion. As a consequence, if our system would 
be layered, we could have a scenario in which 
layered and non-layered architectures coexist; in this 
case, suitable communication protocols between lay- 
ered and non-layered systems are compulsory; their 
realization, however, is often a hard and expensive 
activity. 

• Performance Issues. Several studies point out that 
the number of users accessing e-recruitment systems 
is rapidly increasing [1]. As a consequence, an e- 
recruitment system should be capable of quickly process- 
ing user queries. Such a requirement cannot be easily 
satisfied in a layered system; in fact, in this case, each 
layer supplies services to the layer above it and operates 
as a client for the layer below it. Therefore, in a layered 
system, a continuous message exchange between low- 
level layers and the high-level ones is required. Since 
a message often has to pass through many layers, the 
overhead associated with message exchange is often 
significant and might determine a performance decay. 
Various techniques have been proposed to boost the 
performances of a layered system (e.g., it is possible to 
couple non-adjacent layers) but they seem too difficult 
to be realized in practical contexts, especially in our 
application case where the various layers could belong 
to different owners. 

• Reliability Issues. The fast identification of faults and 
the quick reaction to them cannot be easily performed 
in a layered architecture since a failure in a layer might 



determine the crash of the whole system. Even the (pos- 
sibly non fast) fault identification might be difficult to be 
carried out since the functionalities of the low-level layers 
are often hidden to the high-level ones and, consequently, 
an application running on high-level layers has many 
difficulties in identifying where a certain problem has 
occurred. 

In the following sections we provide a detailed description 
of the two main agents involved in our system, namely the 
User Agent and the Recruitment Agent. 

III. The User Agent 

A. Ontology 

The ontology of the User Agent U Ai, associated with a user 
Ui, stores the profile of Ui. It contains: 

> The code UIDi identifying Ui. 

« A set TopicSeti storing the preferences of uf, each 
preference corresponds to a topic which Ui looked for 
in the past. Each element Topici of TopicSeti is 
represented by a tuple {TopicNamei. , Counti. , First- 
TimeStampi.) , where: (/) TopicNamei- indicates the 
name of Topici ; (ii) Counti. is an access counter 
denoting how many times Topici. has been specified 
in a query submitted by Ui, (Hi) FirstTimeStampi. 
indicates the first time instant in which Topici - has been 
specified in a query of Ui. As it will be clarified in the 
following, these last two coefficients allow the relevance 
of Topici for Ui to be measured. 

• A dynamic set Constraint Seti of constraints that Ui 
fixes for the desired job. Each constraint is described 
by a pair {Feature, Value); some of the possible con- 
straints are the minimum salary Ui would like to earn, 
the city/country where he desires to work, the foreign 
language(s) he knows, his skills, the years of experience 
he has attained, and his academic title(s). 

• A set PastQuerieSi, storing information about the 
queries that Ui posed to the system in the past. Each 
element of PastQueriesi consists of a pair (erf ,af). 
(7i is the satisfaction coefficient; it belongs to the real 
interval [0, 1] and indicates how much Ui appreciated the 
recommendations provided by the system as a response 
to his k*^ query. In the literature many proposals have 
been presented to measure the satisfaction of a user for 
a set of recommendations (see, for example, [19], [20]). 
In this paper we follow the ideas illustrated in [20] and 
define as follows. Let JPList^ be the set of job 
proposals recommended by the system as a response to 
the /c*'* query of Ui and let Accepted J Pf' be the set 
of job proposals accepted by u^; erf can be defined as 

k _ \AcceptedJP^\ 
'^i ~ [jPListfl ■ 

af is the audacity coefficient; it belongs to the real 
interval [0, 1]; the higher af is, the higher the number of 
job proposals returned by the system as its answer to the 
query will be. af is computed by the Recruitment 
Agent; the various strategies for its computation are 
shown in Section IIV-B.II they have been designed in 
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such a way to make our system adaptive against the past 

behaviour of Ui. 

The XML Schema associated with the ontology of U Ai is 
shown in the Appendix. 

B. Behaviour 

The behaviour of U Ai consists of the following steps. 

• Step 1. When m wants to perform a job search, U Ai 
supports him in the construction of a corresponding query 

by means of a suitable wizard. Q\ consists of a pair 
{SelDegree'l, QTSetf). SelDegree'^, belonging to the 
real interval [0, 1], is the Selectivity Degree; it indicates 
how much the system should be selective in the search of 
proposals (see below); the wizard associated with UAi 
proposes to Ui some possibilities and he chooses the 
most suitable one for him in a friendly, guided manner. 
QTSet^ = {QTopicI^, . . . ,QTopic'^J is the set of 
topics describing the job proposals currently interesting 
for Ui. 

Moreover, if m wants to update the constraints for a 
desired job, UAi provides him with a suitable graphical 
interface that allows him to view the constraints he has 
defined, as well as to add, drop or modify some of them. 

• Step 2. UAi updates TopicSeti, this task is performed 
by inserting in it those topics of Q*^ not already present 
therein and by increasing the access counters of the topics 
corresponding to QTapid-^ , . . . , QTopic^ . 

• Step 3. UAi contacts RA and supplies it both Q\ and 
the profile of Wj. RA selects the job proposals answering 
Q'i that best fit the past interests of Ui, as well as 
other interests, someway related to those Ui considered 
in the past and that he still disregarded. Then, it sends 
to UAi both the selected job proposals and the audacity 
coefficient af it used. 

• Step 4. UAi presents to Ui the job proposals provided 
by RA. Ui accepts those ones appearing to be the closest 
to his exigencies and rejects the other ones. After this, 
UAi computes the satisfaction degree that Ui showed 
for the provided recommendations, and stores the pair 
{ai,a^) into PastQuerieSi. 

• Step 5. In order to avoid an excessive growth of 
TopicSeti, U Ai executes a pruning activity on it. Such 
a task is carried out by computing the relevance for Ui of 
each topic Topici- stored in TopicSeti and by remov- 
ing from TopicSeti those topics presenting a relevance 
smaller than a certain threshold. The relevance of Topici . 
at the time instant T is computed by means of the formula 

r{TopiC^^,T) = T-F^rstT^meStamp^^ " ^hlS formuk IS 

justified by considering that the more a user is interested 
in a topic, the more frequently he asks information about 
it. 

In order to show the exploitation of ACML and XQuery in 
our system, in the Appendix, we present the ACML message 
that U Ai sends to RA for requiring a set of recommendations 
and the query that UAi executes for identifying those topics 
of not already present in TopicSeti. 



C. Influences o f job market specifics in the design of the User 

Agent architecture 

The design of our User Agent model has been influenced 
by various job market specifics. In the following we examine 
the most relevant of them. 

1) Construction of a network of job seekers: Several studies 
point out the importance of constructing a job seeker network 
(JSN), i.e., a community of job seekers that share and ex- 
change information about their past job experiences. A JSN 
is an effective channel for disseminating job information and 
can successfully support its users in their job search: as an 
example, a study presented in [21] shows that the probability 
of a user to find a job grows with the increase of the size of 
the network he belongs to. 

Actually, the construction of a JSN is a delicate activity. 
In fact, job seekers, due to both selfishness and rivalry, might 
be unwilling to cooperate. To overcome this drawback, it is 
necessary to create a free-of-competition environment based 
on trustful relationships among its members. In order to carry 
out this task, it is necessary to enhance the cooperation among 
job seekers belonging to the same job context for whom a 
rivalry might not exist; think, for example, to job seekers 
having different experience degrees (e.g., a senior software 
programmer who helps a newly graduated person to find his 
first job). 

The experience shows also that information sharing among 
members of a JSN increases their capability of posing precise 
queries. In fact, users accessing e-recruitment services are ex- 
tremely heterogeneous: job seekers having a deep knowledge 
of a job domain often coexist with other seekers having a su- 
perficial knowledge of it. An expert is capable of formulating 
a query that really captures his needs; on the contrary, a novice 
might ignore the information content of the database storing 
available job proposals and, consequentiy, might be unable to 
formulate precise queries. As previously pointed out, a JSN 
might support novices to compose their queries; specifically, 
a novice might ask the experts of a JSN to support him in his 
query composition. 

Our User Agent model has been designed in such a way to 
facilitate the creation of a JSN; specifically, a User Agent U Ai, 
on behalf of could contact other User Agents for finding 
job seekers having a profile similar to Ui but being more expert 
than him; such a task can be carried out by taking into account 
the lists of topics specified in user profiles and by comparing 
them by means of the Jaccard Coefficient [22]. When the list 
of job seekers similar to Ui is available, Ui can ask U Ai to 
activate a contact with one or more of them in such a way to 
require a support to formulate his queries. 

2} Integration with e-learning systems: Several studies 
point out that the capabihty of a job seeker to continuously 
update his know-how has a significant implication on the 
success of his job search, as well as on the advancement of his 
career; as a consequence, learning activities should be regarded 
as a Ufelong process. For this reason a strict integration 
between e-recruitment and e-learning systems might provide 
a job seeker with a significant competitive advantage. 

A key component of most e-leaming systems is the so- 
called student profile, storing information about the student 
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background knowledge and information needs. In this way, it 
is possible to diversify the learning process on the basis of 
user skills and necessities. 

Our User Agent model can make the interaction between our 
system and an e-learning one easier; specifically, a User Agent 
could transfer the corresponding User Profile to an e-learning 
system that might construct an initial student profile starting 
from it. Conversely, the profile of a student that acquired 
some skills with the support of an e-learning system could 
be exploited to construct an initial user profile for the User 
Agent of our e-recruitment system. 

3) Efficient retrieval of curricula: Presently, a large number 
of curricula vitae is available on the Internet. Generally, a cur- 
riculum consists of a title and a set of additional information 
provided by the job seeker (e.g., his past experience). These 
curricula can be consulted by businesses; a common way to 
do this activity is the so called "curricula spidering", i.e., the 
exploitation of a software agent that tracks down curricula and 
finds those of interest for businesses. 

The search of curricula is generally performed by means of 
a traditional search engine; in this case the business manager 
specifies some words to the engine and this last searches 
for those curricula containing these words. These techniques 
might lead to irrelevant and spurious results; in fact, a cur- 
riculum might be too vague; it might be not updated; last, but 
not the least, the words appearing in a curriculum might not 
belong to the vocabulary of terms used by business managers 
in their searches. 

Our User Agent model is capable of enhancing the curricula 
search capabilities of businesses; in fact: 

• A User Agent is in charge of monitoring the behaviour of 
its user and this allows it to maintain the corresponding 
profile always updated. 

• A User Agent learns the profile of its user on the basis 
of the job proposals that he searches for and, eventually, 
accepts or rejects. In this way, the topics specified in a 
User Profile derive from the job proposal features directly 
inserted by businesses; such a policy avoids ambiguity, 
that could arise when a term present in a traditional 
curriculum could be interpreted in more than one way by 
a business manager, and protects the job seeker against 
his lack of knowledge of the terms that are currently used 
by businesses. 

IV. The Recruitment Agent 

A. Ontology 

The ontology of a Recruitment Agent RA consists 
of a set of job proposals; as pointed out in Sec- 
tion a Job Proposal JPi is described by a tuple 
{JIDi, JURLi, JTopicSeti, JCharacteristicSeti) . 

The XML Schema associated with the ontology of RA is 
shown in the Appendix. 

B. Behaviour 

RA is activated by UAi when Ui wants to perform a query 
for searching job proposals. It receives and the profile 
of uf, it returns to UAf. (i) the job proposals answering 



that best fit the past interests of Ui, as well as other interests, 
someway related to those Ui considered in the past and that 
he still disregarded; (ii) the audacity coefficient it used to 
select the job proposals. In order to perform its activity, RA 
carries out the following steps: 

> Step 1. It queries JPD for retrieving the job proposals 
matching Q\ {keyword-based filtering) and satisfying 
user constraints (feature-based filtering); to this purpose 
it applies classical Information Retrieval techniques [23]. 

> Step 2. It ranks the selected proposals on the basis of user 
preferences, as stored in TopicSeti. Such a task is carried 
out as follows: let JPi = {JIDi, JURLi, JTopicSeti, 
JCharacteristicSeti) be a job proposal and let TSu = 
{Topici- I Topici- G TopicSeti, TopicNamci. G 
JTopicSeti} be the set of topics of J Pi appearing to be 
interesting for u,;. RA assigns an interest degree to J Pi, 
for the user Ui at the current time instant T, according 
to the formula p{TSu,T) = J2 r{Topic,^ , T). 

Topici. eTSii 

Here, the relevance r has been defined in Section IIII-BI 
Clearly, the higher the value of p{TSii, T) is, the higher 
the relevance of JPi for Ui will be. At the end of 
this phase, RA constructs JPTempListf, obtained by 
sorting the selected job proposals on the basis of their 
interest degrees. 

Observe that a more complex formula could be adopted 
for p. However, in this way, the complexity of the Recruit- 
ment Agent would unnecessarily increase. In fact, since, 
during the selection of job proposals, our approach takes 
into account also user past preferences and satisfaction, 
a simple ranking function is sufficient. This is confirmed 
also by the experimental results shown in Section |V] 
JPTempList^ is already a good candidate to be pre- 
sented to Ui. However, two further improvements could 
be performed by RA to make it more adequate. Firstly, 
we observe that J PTempList^ ranks the job proposals 
on the basis of their relevance for Ui, according only to 
his past preferences; however, it does not consider topics, 
someway related to those Ui considered in the past and 
that he still disregarded. Secondly, J PTempList^ could 
still contain quite a large number of proposals. The next 
steps performed by RA are devoted to carry out these 
improvements. 

> Step 3. RA constructs the set SeedProposals^ obtained 
by selecting the first \SelDegree'l x \J PTempList'lW 
proposals of JPTempList'^. The elements of 
SeedProposals^ are used by RA as seeds to choose 
other job proposals containing topics, someway related 
to those Ui considered in the past and that he still 
disregarded. 

• Step 4. RA constructs the final list JPList^ of recom- 
mended job proposals as JPList^ = SeedProposalSiU 
{JPi I JPi G JPTempList^,JP^ G 
SeedProposals^,d{JPi, JPm) < a^}- Here 
d{JPi,JPm) measures the "dissimilarity degree" 
between J Pi and JPm- It is defined as 5{JPi, JPm) — 

^ \.jTopicSeti\+\JToptcSet^\^ ^"^"^^ J opicbcti ana 
JTopicSetm represent the sets of topics associated 
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with J Pi and JPm, respectively, and the symbol \S\ 
represents the cardinality of a set S. Interestingly enough, 
the value ^I'^i^m is known as Dice's coefficient in the 
literature. It is easy to show that 5 ranges between (if 
the two job proposals coincide) and 1 (if the two job 
proposals are completely different). 
The audacity coefficient is dynamically computed by 
RA on the basis of the feedback that Ui showed for 
past recommendations. In this paper we propose three 
different strategies for the computation of this coefficient; 
they are extensively described in Section IIV-B.II 
JPList\, obtained at the end of this step, contains at least 
the seed proposals; moreover, it could include also some 
job proposals relative to topics not particularly relevant 
for Ui in the past, but that could be of interest for him 
in the future (since they are sufficiently similar to those 
he judged relevant in the past). In the selection of this 
last category of proposals 5 plays a key role; in fact, it 
measures the dissimilarity degree of two job proposals on 
the basis of their topics, i.e., of their meaning and seman- 
tics, without considering the relevance that m assigned to 
them in the past. 
• Step 5. RA sends JPList\ and to UAi. 
1 ) Strategies for the audacity computation: In our system 
we have chosen to dynamically update the audacity coefficient 
after each query performed by the user Ui. This allows it 
to be very sensitive to the user judgements about its recom- 
mendations and significantly improves its accuracy. However, 
this makes the manual update of the audacity coefficient 
a very difficult task. In fact, in our opinion, asking the 
user to manually update the audacity coefficient after each 
query is obtrusive. In addition, even with the exploitation of 
suitable graphic interfaces, the user could be not capable of 
understanding the role of the audacity coefficient and to choose 
the most correct value for it. 

However, in our prototype, in order to consider the possible 
presence of very smart users, we have inserted a module 
allowing a smart user to personally modify the values of the 
audacity coefficient determined by the system. 

In this section we illustrate three different strategies that 
may be used for automatically computing the audacity coef- 
ficient that RA associates with Q^. Recall that the higher 
this coefficient is, the higher the number of job proposals 
returned by the system for answering will be; this strictly 
depends on the satisfaction that Ui showed for the recommen- 
dations of RA in the past. As a consequence, any strategy for 
the computation of a\ must evaluate the satisfaction of Ui w.r.t. 
the previous recommendations of RA. As previously pointed 
out, the choice to make audacity dependent on satisfaction 
allows our system to be adaptive against the past behaviour of 

Ui. 

a) Strategy 1: Positive and Negative Feedback (PN¥). 
In this strategy, a\ is dynamically updated on the basis of 
the feedback provided by Ui for the last recommendation; this 
is encoded in the satisfaction coefficient (J^~^ relative to the 
recommendations associated with the query Q^~^. Recall that 
<J^^^ is the fraction of job proposals accepted by Ui w.rt. 
those recommended by the system when it answered 



PNF works as follows: 

> If, during the execution of Q^^^, the number of accepted 
recommendations has been greater than the number of the 
rejected ones (i.e., a^~^ > then it is possible to ai-gue 
that Ui has appreciated the system recommendations, 
since he accepted most of them. As a consequence, if 
the system would send further recommendations, it is 
extremely probable that Ui would receive other proposals 
interesting for him. Now, since in our system the amount 
of proposals sent to the user as the answer to the query 
is strictly related to the audacity coefficient a*^, increasing 
this coefficient would imply an increase of the number of 
proposals recommended by the system when answering 

Q'i- 

Clearly, the key issue in this reasoning is the correct 
estimation of the increase of af. In fact, if such an 
increase is too high, then Ui would receive several use- 
less proposals and his perceived quality of the overall 
recommendations could decrease. On the contrary, if the 
increase is too low, the system could miss to recommend 
some proposals interesting for Ui. The previous reasoning 
led us to define the increase to apply on the audacity 
a'^~^ for obtaining as a linear function of the user 
satisfaction; formally, e'^ — a^^^ — ^. This formula shows 
that the more Ui is satisfied, the more (t*^^^ is higher 
than i and the more a'^ is increased w.r.t. a'l^^. Clearly, 
other functions could be adopted for e*^, e.g., logarithmic, 
quadratic, cubic or exponential functions; however, in our 
opinion, the linear function is the most suitable one to 
satisfy the contrasting exigencies mentioned above and 
to provide the system with the correct balance between 
stability and readiness in adapting its behaviour to user 
satisfaction (see the Appendix for an experimental anal- 
ysis of this topic). 

• If, during the execution of Qi^^, the number of rejected 
recommendations has been greater than the number of the 
accepted ones (i.e., ~^ < 5), then it is possible to argue 
that Ui has disliked system recommendations, since he 
refused most of them, and that he is interested to examine 
fewer proposals during the next queries. As previously 
pointed out, the number of system recommendations 
depends on ; therefore, in order to make the system to 
recommend fewer proposals to Ui, it is necessary that 

is lower than af^^. For the same reasoning introduced in 
the previous case, the most suitable function for defining 
the decrease e*^ to apply to a*^~^ for obtaining af appears 
the linear one. More formally, the decrease can be 
defined as e*^ = i - crf^^ 

• The last possible situation happens when the number 
of accepted proposals has been equal to the number 
of the rejected ones (i.e., crf^^ = ^). This situation 
can be seen as a specific case of both the first and 
the second situations described above. If it is consid- 
ered as a particular case of the first (resp., second) 
situation, the increase (resp., decrease) is equal to 
and, consequently, it is possible to conclude that the 
number of recommendations should be neither increased 
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nor decreased; this implies that the audacity coefficient 
should be maintained constant. 
The previous reasoning leads to the following formula for 
computing the new value of : 

r min{l, a^'^^ +ef} if crf^^ > i 

[ max{0, - if ^t' < k 

This formula is valid when Ui exploits the system at least 
for the second time. During the first query a] is set equal to 
a constant value a. An analysis of the impact of a on the 
system performance is reported in the Appendix. 
b) Strategy 2: 2-Least Square Error (2-LSE). 

This strategy is based on the Least Square Error tech- 
nique (LSE) that is largely exploited in several research 
areas, such as manufacturing engineering [9], bioinformatics 
[10], econometrics [11], and so on. LSE considers two sets 
X = {a:;i,a;2, . . . ,a;Ar} and Y = y2, • ■ • , yw} and a 
function y — f{x, oo, ai, . . . , Up) depending on p+1 unknown 
parameters but having a predefined form (e.g., / might be a 
polynomial, a linear combination of exponentials, a sinusoid, 
and so on). The goal of LSE is to identify the values Oq, a*, 
. . ., a* of the parameters ao, ai, . . . , such that the residual 

N 2 

R = J2 {yi ^ fi^h^oi O'l, ■ ■ ■ : ap) is minimum. In such a 

1=1 

case it is said that /(x, Oq, a^;, . . . , a*) is the function that best 
"fits" the sets X and Y, in the least squares sense. 
In order to compute a'^, this strategy: 

• Applies the LSE technique for determining the values 
Cq , . . . , a* of the parameters aq , . . . , ftp of the fitting 
function. To this purpose, it exploits the sets X = 
K,af,...,a^l} and Y = {a} ,al . . . ^af}- Ob- 
serve that, as a consequence of this choice for the sets X 
and Y, the fitting function represents the user satisfaction 
against the system audacity. 

• Sets a'H to the value of the audacity corresponding to the 
maximum value0 of the fitting function and, consequently, 
to the maximum value of the user satisfaction. 

A crucial problem in the implementation of the LSE strategy 
is the choice of both the shape and the parameters of the fitting 
function. Such a choice must minimize the residual R; as a 
consequence, it requires to compute the partial derivatives of 
R and to solve the set of equations — 0,yj = 0, . . . ,p. 

If we use, as fitting function, a combination of sinusoids, 
logarithms and exponentials, these equations generally define a 
non-linear system that, often, cannot be exactly solved; in this 
case, even the calculation of an approximate solution requires 
an heavy computational effort. 

On the contrary, if the fitting function has a polynomial 
shape, i.e., f{x, ao, oi, . . . , Op) — aax^ + aix^^^ + . . . + a.p, 
it is possible to show that these equations are equivalent to 
the linear system [24] Ax = b, where A is a matrix called 
Vandermonde matrix. It is possible to show that at least an 
approximate solution of this system can be computed in a 

^Observe that, since the independent variable (i.e., the audacity) belongs to 
the compact interval [0, 1], the fitting function has always a maximum within 
this interval. 



polynomial time; generally, the exact solution can be computed 
in 0{p^) steps [24]; as a consequence, a polynomial shape for 
the fitting function appears well suited for our purposes. 

The next step of our activity consists of determining the 
order of the polynomial to be used as fitting function. To this 
purpose observe that LSE must compute the audacity value 
that maximizes the fitting function f{x); to compute this value 
we need to determine the derivative f'{x) of f{x) and to 
solve the equation f'{x) — 0. Now, if f{x) is a p*'' order 
polynomial, f'{x) will be a {p — 1)*'* order polynomial; as 
a consequence, we have that: (i) if p — 2 (i.e., if the fitting 
function is a parabola), it is possible to directly obtain an 
exact solution of f'{x) = by solving a first order equation; 

if p = 3 (i.e., if the fitting function is a cubic), it is 
possible to obtain an exact solution of f'{x) = by solving 
a second order equation; ( Hi) if p > 3, the computation of the 
roots of f'{x) can be performed by means of numerical (and 
approximate) techniques; with regard to this, it was shown that 
the time necessary for finding all the roots of f'{x) — with 
a maximum error equal to is 0{p\ogp'^ + plog^ plogb) 
[25]. 

This analysis shows that, for obtaining a correct (i.e., not 
approximate) solution of the equation f'{x) — 0, the fitting 
function must be either a parabola or a cubic. Since the 
accuracy ensured by the polynomial of degree 2 (i.e., the 
parabola) is high [26], and since the computation time it 
requires for solving f'{x) = is lower than that necessary 
if the fitting function is a cubic, we have adopted the parabola 
as fitting function; in this case the LSE technique is called 
2-LSE technique. 

Therefore, in our case, the fitting function is y = aox"^ + 
aix + a2\ this choice implies that the 2-LSE strategy is valid 
only when Ui exploits the system at least for the fourth time. 
During the first three queries a},af, are set equal to three 
constant values a', a", a'", respectively. An analysis of the 
impact of a', a", a'" on the system performance is reported 
in the Appendix. 

c) Strategy 3: Weighted Sum (WS). 

In order to compute a^, this strategy applies the PNF 
and the 2-LSE strat^ies and obtains two audacity values 
a, ' and a,' ^ . The overall audacity value is 
obtained by computing a suitable weighted mean of ' 

J k,2-LSE r- „ k k.PNF , /-, n 

and a- ; specifically, a,; = 7 x a + (1 — 7) x 

k 2— LSE 

ttj ' . Here, 7 is a coefficient ranging in the real interval 

[0, 1]. Specifically, if 7 = 1, the WS strategy coincides with 
the PNF strategy; on the contrary, if 7 = 0, the WS strategy 
coincides with the 2-LSE strategy. In our experiments we have 
considered different values for 7 and we have studied their 
impact on the system performance (see the Appendix for more 
details); in addition, we have examined how to dynamically 
compute 7 for maximizing the system performances. 

V. Experimental Results 

In this section we describe in detail the various experiments 
we have conducted for testing the performance of our system. 
Specifically, in Section IV-AI we describe the characteristics 
of both the users and the job proposals considered in our 
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tests; Section IV-BI presents the accuracy measures we have 
chosen for testing our approach. Section IV-CI is devoted to 
make a comparison of the three strategies for the audacity 
computation. In Section IV-DI we evaluate the performance of 
our system in different appUcation domains. Finally, Section 
IV-EI is devoted to present an experimental comparison of our 
system with other e-recruitment ones. 

In order to perform our tests, we have designed a pro- 
totype implementing our approach. This prototype has been 
developed in JADyj (Java Agent DEvelopment framework), a 
software framework conceived for supporting the implemen- 
tation of agent-based applications in compliance with FIPA 
specifications [27]. 

A. Characteristics of the users and job proposals 

In our tests we have considered a set of users U Set = 
U2, ■ • ■ , U200} consisting of 200 volunteers. As for their 
distribution against their age we have that: (i) 21.00% of them 
was 18-24 years old; (ii) 39.00% of them was 25-34 years 
old; (Hi) 25.50% of them was 35-44 years old; (iv) 13.50% of 
them was 45-54 years old; (v) 1.00 % of them was more than 
54 years old. 

As for their distribution against their past usage to com- 
mercial e-recruitment systems we have that: (i) 11.00% of 
them had never used a commercial e-recruitment system; (ii) 
10.00% of them exploited it very rarely; (Hi) 10.00% of them 
used it rarely; (iv) 25.50% of them exploited it sometimes; (iv) 
31.00% of them exploited it often; (v) 12.50% of them used 
it very often. 

The available job proposals, extracted from 
the sites |http : //www . jobpilot ■ CO ■ uk and 
|http : / /www ■ careerbuilder . coin| were about 3000. 
They belonged to different domains; specifically, reference 
domains where: (i) "Information Technology" for 19.29% of 
them; (ii) "Health Care Management" for 17.02% of them; 
(Hi) "Finance" for 15.84% of them; (iv) "Engineering" for 
12.07% of them; (v) "Banking" for 10.04% of them; (vi) 
"Legal" for 7.02% of them; (vii) "Real Estate Management" 
for 5.26% of them; (viii) "Others" for 13.46% of them. 

B. Evaluation Measures 

In our tests we have computed two widely accepted eval- 
uation measures, namely Precision and Recall. Precision is 
defined as the share of job proposals accepted by Ui among 
those recommended by the system; Recall is the share of job 
proposals suggested by the system among those of interest for 

Ui. 

These parameters have been computed as follows: 

• Each user Ui e USet was asked to submit a query Q\ 
and the corresponding JPTempList^ was obtained (see 
Section HVli. 

• Ui was asked to identify the subset UserListf C 
JPTempList^ of job proposals that he considered in- 
teresting. 

^JADE is an open source platform; it can be downloaded from 
|http : // Sharon . cselt ■ it/pro jects/ jade| . 



^ AvgPre PNF I 
^ AvgPfe 2-LSE 




qI 1 1 1 1 1 

5 10 15 20 25 

Number at Queries 

Fig. 2. Average Precision for tlie PNF and the 2-LSE strategies 

• Our prototype was run for obtaining the set JPList\ C 
J PTempList'l of job proposals to be recommended to 

Ui. 

• The Precision Pre^ and the Recall Rec^ relative 
to the fc*'* query have been obtained by applying 
the formulas: Pre^ — ,l Reef = 

^ \JPList^\ ' * 

IJPList'^nUserList'^l 
[UserList'^ \ 

m Finally, the Average Precision AvgPre'^ and the Average 
Recall AvgRec^ relative to the k^^ query have been 

V _ Pre'' 

obtained as AvgPre'' = I'^get]' ^vgRec'' = 

Z-^i=l.-\USst\ ' 

\USet\ 

Observe that Precision, Recall, Average Precision and Av- 
erage Recall belong to the real interval [0, 1]; specifically, the 
higher these coefficients are the better the system works. 

C. Experimental comparison of the three strategies imple- 
mented in our approach 

In this section we experimentally compare the strategies 
for the audacity computation described in Section IIV-B.II 
Actually, it is necessary to examine only the PNF and the 
2-LSE strategies; in fact, the WS strategy is a weighted mean 
of the two other ones. In the test of the PNF strategy we have 
set a — 0.55 whereas in the test relative to the 2-LSE strategy 
we have set a' = 0.5, a" ~ 0.6, a'" — 0.4 (see the Appendix). 

For each strategy we have computed the Average Precision 
and the Average Recall against the number of queries. The 
obtained results are shown in Figures |2] and [3] From the 
analysis of these figures we can conclude that: 

• If the number of queries carried out by users is low 
(i.e. lesser than 10), the PNF strategy shows a better 
performance than the 2-LSE strategy; in fact, its Average 
Precision and its Average Recall increase up to 0.75 and 
0.69, respectively. This behaviour can be motivated by 
considering that, before 10 queries have been performed, 
the 2-LSE strategy has still not completed the audacity 
"tuning" and, hence, the corresponding results are not 
particularly satisfactory. 

• If the number of queries carried out by users is high 
(i.e. greater than 15), the performance improvements of 
the PNF strategy are small. On the contrary, the 2-LSE 
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AvgRec PNF I 
^ AvgRec 2-LSE 
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Number at Queries 

Fig. 3. Average Recall for the PNF and the 2-LSE strategies 

Strategy shows a very good performance; as an example, 
after 15 queries, the Average Precision and the Average 
Recall are 0.85 and 0.76, respectively. This behaviour 
can be motivated by considering that the 2-LSE strategy 
"tunes" the audacity coefficient on the basis of the user 
feedback during all the fc — 1 previous queries, whereas 
the PNF strategy considers only the last one. As a 
consequence, a high number of queries allows the 2-LSE 
strategy to more precisely predict user expectations. 

• During the initial queries, both Average Precision and 
Average Recall have an oscillatory behaviour; this de- 
pends on the fact that, initially, both user profiles and 
user feedbacks are relatively poor 

• Finally, we observe that the Average Precision is gen- 
erally greater than the Average Recall; this confirms the 
results obtained in [28]. As pointed out in this paper, such 
a feature is to be considered a positive characteristic for 
a recruitment system; in fact, in this application context. 
Precision is generally assumed to be more important 
than Recall since the user could be frustrated by many 
irrelevant proposals. 

Finally, it is worth pointing out that the results of this 
experiment confirm the results of the experiments about the 
tuning of 7 described in the Appendix. In fact, in that case, 
we have obtained that, for a small number of queries (i.e., 
lesser than 10), the value of 7 should be high (i.e., the WS 
strategy should tend to coincide with the PNF strategy); on 
the contrary, when the number of queries is high, the value of 
7 should be low (i.e., the WS strategy should tend to coincide 
with the 2-LSE strategy). 

D. Performance of our system in different application domains 

This series of experiments was devoted to determine the 
performance of our system in several application domains; 
some of them were very generic, other ones were more 
specialized. 

In all existing e-recruitment systems, job proposals can 
be hierarchically organized on the basis of the application 
domains they refer to; specifically, the most generic domains 
(e.g., "Health-Care" or "Information Technology") are located 
on the top of the hierarchy; on the contrary, the most spe- 
cialized ones (like "Biochemical Scientist") are located at 
the bottom. This hierarchy can be graphically represented by 



means of a tree; each node of the tree, with the exception of the 
root, represents a domain; a fragment of this tree is graphically 
shown in Figure |4] With regard to this classification tree, we 
say that the specialization level of a domain D is j if the 
depth of the node associated with D is j. As an example, the 
specialization level of the domain "Application Developer" is 
3, since the depth of the node representing this domain is 3. 

In this test we have computed the Average Precision and 
the Average Recall obtained by our system when applied to 
four domains, namely "Information Technology", "Pharmacy", 
"Software Support" and "Biomedical Scientist", character- 
ized by different specialization levels. During this compu- 
tation we have used the WS strategy by setting j{k) = 
max {0,1 - {^)}, a = 0.55 and ^ = 0.5, = 
0.6, a'" = 0.4. The obtained results are shown in Table U 

From the analysis of this table it is possible to observe 
that, in general domains (i.e., domains at a specialization 
level 1 or 2), our system rapidly (i.e., after a few number 
of queries) obtains good results; however, this performance 
slightly worsens when the number of queries increases. On 
the contrary, in specific domains, our system needs several 
queries for achieving good results but, after this initial phase, 
it maintains, and even improves, its performance. 

This behaviour can be motivated by the following reasoning: 
during the first queries, users interested in general domains do 
not have a precise idea about their needs and, consequently, 
many of the provided recommendations appear interesting to 
them. On the contrary, users interested in specific domains 
have a precise idea of their desires already during the initial 
phase; as a consequence, it is more difficult to satisfy them 
during the first queries. When the number of queries increases, 
users of general domains "ripen" a more precise idea of their 
needs, but the application domain they are interested in is 
too generic to precisely satisfy their requirements; however. 
Table H] shows that, even in these conditions, our system 
achieves quite good results. On the contrary, after many 
queries, the profiles of the users of specific domains are quite 
rich and this allows our system to precisely identify their needs 
and to significantly reduce the job proposals search space. 

E. Experimental comparison of our system with other e- 
recruitment prototypes 

We have experimentally compared our system with 
other, already available, ones. Specifically, the systems 
that we have taken into account were: (i) CareerBuilder - 
www.careerbuilder.com, (ii) Ijob - www.ljob.co.uk, 
(Hi) Job Search - www.jobsearch.co.uk, (iv) 
JobPilot - www.jobpilot.co.uk, and (v) Fish4Jobs - 
www.fish4.co.uk/iad/jobs. These systems share various 
similarities; specifically, they operate on the Internet and 
manage an internal, relational database storing all available 
job proposals. In addition, they provide job seekers with 
a graphical interface to retrieve the job proposals they are 
interested in; a job seeker can use this interface to specify 
some constraints (e.g., the place where he would like to work), 
as well as a list of keywords describing the job typology 
of his interest. Finally, some of these systems allow each 
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Fig. 4. A fragment of a tree classifying job proposals 



TABLE I 

Performance of our system in different application domains 



Domain 


Specialization Level 


Que,y 5 


Query JO 


Quen' 15 


Query 20 


Que>y 25 


Avg Pre 




Avg Pre 


Avg Rec 


Avg Pre 


Avg Rec 


Avg Pre 


Avg Rec 


Avg Pre 


Avg Rec 


Information Technology 


1 


0.73 


0.67 


0.76 


0.70 


0.74 


0.68 


0.72 


0.65 


0.68 


0.61 


Pharmacy 




0.68 


0.63 


0.72 


0.69 


0.79 


0.71 


0.78 


0.70 


0.77 


0.69 


Software Support 


3 


0.54 


0.52 


0.61 


0.58 


0.80 


0.72 


0.85 


0.73 


0.86 


0.75 


Biomedical Scientist 


4 


0.49 


0.46 


0.60 


0.56 


0.81 


0.73 


0.86 


0.76 


0.90 


0.82 



user to construct a simple profile by filling a questionnaire; 
each profile stores user skills (e.g., the foreign languages he 
knows) as well as a brief curriculum vitae. Since the queries 
submitted by a job seeker consist of a list of keywords, 
each system must transform them into SQL queries; for this 
reason, the underlying DBMS must incorporate Information 
Retrieval (hereafter, IR) algorithms [29]. 

Our activity aimed at comparing the accuracy of our system 
against that of the other systems mentioned previously. To this 
purpose we asked each user of USet to submit some queries; 
these were processed by the systems into examination and 
the corresponding Average Precision and Average Recall were 
computed. As for our system we have chosen the WS strategy 
and we have set j{k) — max {0, 1 — (-35^) }, a = 0.55 and 
a' — 0.5, a" = 0.6, a'" = 0.4. The results we have obtained 
are shown in Table HI] From the analysis of this table, we can 
observe that the accuracy of our system is quite satisfying. This 
interesting result can be justified by the following reasoning: 

• In order to score available job proposals, the ranking 
function adopted by our system considers several aspects 
of the profile of a user, as well as his reaction to past 
proposals (see Sections IlII-BI and lIV-BI l; on the contrary, 
the ranking functions adopted by the other systems into 
examination consider only some constraints specified by 
the user (e.g., the salary he would like to earn). Taking 
into account a large number of issues makes our system 
more sensible to the real user exigencies; this justifies 
the improvement of the Average Precision w.r.t. the other 
systems into examination. 

• Our system suggests to users not only the job proposals 
exactly matching their queries but also those ones some- 
way semantically related to them. Hence, it is capable 
of suggesting potentially interesting job proposals that 
cannot be revealed by traditional IR algorithms; this 
implies an improvement of the Average Recall w.r.t. the 



other systems into examination. 
> Our system is capable of considering job proposals avail- 
able in several other systems; this contributes to improve 
both the Average Precision and the Average Recall w.r.t. 
the other systems into examination. 

However, neither Precision nor Recall alone are capable of 
completely capturing the "quality" of the proposals provided 
by e-recruitment systems. As an example, a key aspect of 
these systems regards their capability of correctly scoring 
their job proposals. To better clarify this concept, consider 
two e-recruitment systems, ERSi and ERS2, that receive 
the same query from a user Ui and return the same set of 
results but in a reverse order, in other words, ERSi returns 
the list (JPi, JP2, ■ • ■ , JPn-i, JPn) whereas ERS2 returns 
the Ust (JP„, JP„-i, . . . , JP2, J Pi); this impUes that ERSi 
(resp., ERS2) rates JPi (resp., JP„) as the most relevant 
proposal for Ui and JP„ (resp., JPi) as the least relevant 
one. Finally, assume that Ui fully agrees with the suggestions 
provided by ERSi. In this case ERSi and ERS2 return the 
same proposals and, consequently, obtain the same Average 
Precision and the same Average Recall; however, from the 
previous reasoning, it is possible to conclude that the "quality" 
of suggestions provided by ERS2 is lower than the quality of 
suggestions provided by ERSi: in fact, Ui must browse the 
whole list of suggestions provided by ERS2 to find the job 
proposal of his maximum interest and this might be both a 
boring and a time-consuming activity. 

To measure the ranking capability of an e-recruitment sys- 
tem we adopted the Newell Distance [30], a parameter defined 
in User Modelling theory. To define the Newell Distance 
associated with a user Ui and a system to evaluate Sj we 
need to consider a set JPSet of test job proposals and a 
pair of functions, sys and usr; sys (resp., usr) takes a job 
proposal JPk G JPSet as input and returns an integer p 
belonging to the set {1, 2, ... , \JPSet\} as output; p indicates 
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TABLE II 

Average Precision and Average Recall of the systems into examination 



Syslem 


Q„e,y 5 


Queiy 10 


Q„e,y 15 


Query 20 


Query 25 


Avg Pre 




Avg Pre 


Avg Rec 


Avg Pre 


Avg Rec 


Avg Pre 




Avg Pre 


Avg Rec 


Our Syslem 


0.62 


0.61 


0.73 


0.66 


0.81 


0.74 


0.86 


0.76 


0.87 


0.79 


CareerBuilder 


0.50 


0.48 


0.61 


0.52 


0.68 


0.57 


0.72 


0.63 


0.73 


0.65 


Ijob 


0.52 


0.46 


0.63 


0.57 


0.71 


0.66 


0.68 


0.67 


0.70 


0.63 


Job Search 


0.54 


0.44 


0.64 


0.60 


0.69 


0.64 


0.70 


0.65 


0.73 


0.68 


JobPilot 


0.44 


0.51 


0.47 


0.53 


0.59 


0.54 


0.67 


0.60 


0.71 


0.66 


Fish4Jobs 


0.47 


0.48 


0.50 


0.52 


0.60 


0.55 


0.63 


0.62 


0.69 


0.63 



TABLE III 

Average normalized Newell distance achieved by the systems 
into examination 



Syslem 


Que,y 5 


Query 10 


Quer,- 15 


Query 20 


Query 25 


Our System 


0.27 


0.20 


0.13 


0.09 


0.06 


CareerBuilder 


0.40 


0.31 


0.19 


0.15 


0.12 


Ijnb 


0.38 


0.36 


0.22 


0.14 


0.11 


Job Search 


0.33 


0.29 


0.21 


0.14 


0.09 


JobPilot 


0.39 


0.33 


0.29 


0.24 


0.15 


Fish4Johs 


0.38 


0.31 


0.25 


0.19 


0.13 



that, according to Sj (resp., Ui), JPk has the p highest score 
among all job proposals existing in JPSet. 

ociated w 

w{usr{J Pk)) X usr{JPk 



The Newell Distance A/ij associated with Ui and Sj is 



defined as A/ij — 



E 

JP^SJPSet 

w(sys{JPk)) X sys(JPk) |. Here w{i) is a weighting function 

\JPSet\- 



defined as: w(i) = ^- 



it assumes its maximum 
value when i — 1 and decreases when i increases; this implies 
that it gives a different importance to the errors possibly made 
by the system to evaluate; specifically, if a system is unable 
to detect the most important job proposal, it commits a more 
serious error than if it is not capable of identifying the least 
relevant ones. 

The Newell Distance can be properly normalized in such 
a way to lie in the real interval [0,1]; specifically, if we 
indicate by TV™"^ the maximum value of the Newell Distance 
measured over all users and systems into consideration, the 
normalized Newell Distance can be defined as TVi, — 



Observe that the lesser A/ij is, the better a system works. 

In Table Hill we report the normalized Newell Distance, aver- 
aged on all users, obtained for the systems into examination. 
From the analysis of this table we observe that the Newell 
Distance obtained by our system is smaller than that obtained 
by the other systems into evaluation. In our opinion, this result 
is motivated by considering that: 

• The parameters adopted by our system for scoring avail- 
able job proposals are strictly related to the profile of 
a user, as well as to his reaction to past proposals; this 
enhances its capability of correctly identifying the most 
relevant job proposals. 

• Our system continuously monitors user behaviour and is 
capable of detecting if the relevance of a job proposal 
for a given user changes over time. On the contrary, the 
other systems we have considered in our experiments do 
not consider the possible modifications of user desires 
over time. 

From the previous experiments we can conclude that the 
recommendations provided by our system are more accurate 
than those supplied by the other systems into examination; this 




Number of Queries 



Fig. 5. Average size of user profiles against tlie number of queries canied 
out by users 



improvement is mainly due to quite a sophisticated manage- 
ment of information about user profiles and past behaviour. 
This information is combined with classical IR techniques 
exploited also by the other systems into consideration and 
allows the most appropriate answers for user needs to be 
found. 

On the other hand, our system requires an additional amount 
of space for storing user profiles; this is a problem affecting 
all systems handling and exploiting user profiles. In order to 
quantify the space requirements of our system for storing user 
profiles we have performed a final experiment. Specifically, 
we have computed the average size of user profiles (expressed 
in Kbytes) against the number of queries carried out by users. 
The obtained results are shown in Figure |5] 

From the analysis of this figure we can observe that, 
initially, user profiles are generally "poor" and, consequently, 
the storage space they require is negligible; when a user poses 
his queries, the system enriches the corresponding profile by 
inserting new topics. After a "reasonably large" number of 
queries (i.e., after about 10 queries) user profiles become 
rich and occupy a certain amount of space; however, this 
space occupation does not increase during the next queries; in 
fact, when user profiles become excessively large, the system 
activates a pruning task in such a way that the number of 
new topics inserted in a profile is approximately equal to the 
number of topics removed from it; as a consequence, after 15 
queries, the average size of user profiles remains quite constant 
(specifically, it is about 10 Kbytes). 

The previous analysis shows that the quantity of space our 
system needs for storing user profiles is limited and acceptable; 
moreover, since user profiles are implemented in XML, it is 
possible to apply the methodologies illustrated in [31] for 
handling their space occupation in a more efficient way. 
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VI. Related Work 

Even if e-recruitment is quite a novel research area, sev- 
eral systems devoted to handle such an activity have been 
already presented in the literature. In this section we aim at 
positioning our system amongst other related ones. In order 
to carry out this activity, we have considered three terms of 
comparison, namely: (i) Purpose: e-recruitment systems can 
be classified as company oriented, if they aim at supporting 
companies in the selection of new candidates, and seeker 
oriented, if they support users looking for new job proposals; 
(ii) Architecture: e-recruitment systems could be centralized, 
if a single computational entity is in charge of managing all 
activities, distributed, if tasks are partitioned among several 
computational entities, and mixed, if some tasks are performed 
in a centralized fashion and other ones are carried out in a 
distributed way; ( Hi) User Profile Construction: e-recruitment 
systems might be obtrusive, if they interact with the user for 
constructing his profile, or unobtrusive, if they learn the profile 
of a user by monitoring his accesses to the system. 

On the basis of these terms of comparison our system can 
be considered seeker oriented, mixed and unobtrusive . 

In the following we compare our approach with other 
related ones and highlight similarities and differences existing 
among them. Table |IV] provides a summary of the in-depth 
comparison we carry out in the following subsections. 

A. Approach of [32] 

[32] proposes a recommender system devoted to support 
users looking for new job proposals. The system operates as 
follows: (i) it classifies each user according to his personal 
traits (e.g., shy/sociable or talktive/taciturn); (ii) a user queries 
it to retrieve new job proposals; (Hi) it ranks retrieved job 
proposals according to their similarity to the personal traits 
of the user. There are some similarities between our approach 
and [32]; in fact, both of them: (i) are seeker oriented; (ii) 
can cooperate with existing e-recruitment Web sites; (Hi) 
classify job proposals according to their relevance for the 
user As for differences, we may notice that: (i) in [32] user 
profiles are constructed by means of psychological tools (e.g., 
questionnaires) whereas our system obtains information about 
users unobtrusively, by watching their interactions with it; ( ii) 
in [32] no mechanisms for user profile update are provided; 
( Hi) the architecture of [32] is centralized whereas our system 
is mixed, since the Recruitment Agent performs most of the 
activities connected with the selection of the job proposals 
best fitting user exigencies, but User Agents, Wrapper Agents 
and Company Agents continuously cooperate with the Re- 
cruitment Agent for performing the whole recruitment task 
(see SectionHJl. Interestingly enough, our system might inherit 
some features typical of the approach of [32]; specifically, the 
user profiles managed in our system might include information 
about user personal traits, analogous to that handled in [32], 
to enhance the job search process. 

B. Approach of [33] 

In [33] the authors present a probabilistic e-recruitment 
system that uses both content-based and collaborative filter- 



ing techniques to produce recommendations. There are some 
similarities between our approach and that described in [33]; 
more specifically, both of them: (i) provide a mechanism for 
automatically and unobtrusively learning user preferences; (ii) 
take into account the user feedback that influences the system 
behaviour: specifically, in our approach, the knowledge of 
the job proposals accepted/rejected by the user is exploited 
for automatically tuning the system audacity; analogously, 
in [33], user preferences influence the coefficients of the 
probabilistic model adopted for deriving recommendations; 
(Hi) are seeker oriented. As for differences, we may notice 
that: (i) [33] exploits a probabilistic model for deducing new 
recommendations; such an approach is very refined but it 
might be excessively time-consuming; (ii) our system is mixed 
whereas the approach of [33] is centralized; (Hi) the approach 
of [33] is hybrid, whereas our own is content-based. 

C. CASPER 

In [28], [34] flie system CASPER is described. CASPER is 
a client-server system that separately exploits collaborative 
filtering and content-based techniques to provide new job 
recommendations. There are some similarities between our 
approach and CASPER; in fact, both of them: (i) are seeker 
oriented; ( ii) construct user profiles by unobtrusively monitor- 
ing user activities; (in) follow similar criteria for estimating 
user preferences; (iv) are mixed. As for differences we may 
notice that: (i) CASPER can operate either as a content- 
based or as a collaborative filtering recommender system; 
on the contrary, our system is content-based; (ii) CASPER 
does not provide tools for modelling the semantics of a job 
proposal; on the contrary, our approach uses an XML Schema 
for representing the main features of each job proposal; these 
features play a key role during the recommendation process. 

D. Personnel Mall 

In [35] the Personnel Mall system is proposed. Personnel 
Mall is an agent based platform conceived for matching job 
seekers with companies in a distributed (and electronic) job 
marketplace. Specifically, Personnel Mall defines a set of rules 
to represent and manage the preferences/exigencies of job 
seekers and companies and uses them during the matching 
process. There are some similarities between our approach 
and Personnel Mali, in fact, both of them: (i) use the agent 
technology to perform e-recruitment activities; (H) model job 
marketplace as a complex system in which heterogeneous 
components (i.e., companies and individuals) incessantly in- 
teract. As for differences, we may observe that: ( i) Personnel 
Mall is company oriented; ( ii) in Personnel Mall the matching 
between a job seeker and a job proposal depends not only 
on subjective variables (e.g., the personal interests of a user) 
but also on economic parameters (e.g., the quantity of labor 
that companies would like to hire for a certain wage and the 
quantity of labor that a job seeker would like to supply); 
(Hi) Personnel Mall aims at maintaining the conditions of 
equilibrium in the job marketplace: as an example, it is capable 
of dynamically raising/lowering wages in response to labor 
surpluses/shortages; such a feature is not present in our system; 
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TABLE IV 

A COMPARISON OF SOME E-RECRUITMENT SYSTEMS 



.Sy.vrf/» 1 1 furpo^e \ Archileclure \ U.^er Profile cofislfuction \ Aiidilioiw! Fmhires 



Our approach 


Seeker Orienied 


Mixed 


Unobtrusive 


It exploits Agent Technology and XML 


Supjarcrndcv cl id. |.12| 


Seeker Oriented 


Centralized 


Obtrusive 


It considers the personal traits of a user 


i-arher et id. 


Seeker Oriented 


Centralized 


Unobtrusive 


It is an hybrid recommender system 


CASPER [28], [34] 


Seeker Oriented 


Mixed 


Unobtrusive 


It separately exploits content based and collaborative filtering techniques 


Personnel Mall [35] 


Company Oriented 


Distributed 


Obtrusive 


It exploits the agent technology and a matching algorithm 


Dafoulas et at. [36] 


Company Oriented 


Centralized 


Obtrusive 


It allows online interviews to be performed 


CommOnCV [37] 


Seeker Oriented 


Centralized 


Obtrusive 


It exploits Semantic Web languages to model a curriculum vitae 


e-SRS [38] 


Company Oriented 


Distributed 


Obtrusive 


It exploits both psychometric techniques and clustering algorithms to recruit candidates 



( iv) Personnel Mall does not provide mechanisms for updating 
the preferences and the exigencies of job seekers; (v) Personnel 
Mall is obtrusive; (vi) Personnel Mall is distributed; on the 
contrary, our system is mixed. 

E. Approach of [36] 

In [36] an agent based approach for supporting e- 
recruitment activities is proposed. This system embodies the 
functionalities typical of a traditional e-recruitment Web site 
(e.g., a user can insert/update his profile, a company can post 
new job proposals, and so on); moreover, it allows companies 
to perform an on-line interview of candidates; this interview 
is useful for both companies (that can perform a preliminary 
screening of candidates) and job seekers (that can enrich their 
profiles by storing their past interviews). It is possible to 
identify only few similarities between our approach and [36]; 
specifically, both of them: (;) define and manage user profiles; 
(ii) use the agent technology. As for differences we observe 
that: ( ;) [36] is company oriented; ( //) [36] provides a layered 
and centralized architecture whereas our approach is mixed; 
(Hi) [36] is obtrusive. 

F. CommOnCV 

In [37] the authors propose CommOnCV, a system capable 
of representing and managing a curriculum vitae by means 
of Semantic Web tools. The approach implemented by Com- 
mOnCV is based on the concept of competency that is used 
to represent the knowledge/skills owned by a job seeker; 
competencies are defined and described by means of suitable 
ontologies. In addition, CommOnCV defines a curriculum vitae 
as a network of competencies and represents it by means of Se- 
mantic Web languages, like RDF/RDFs or DAML+OIL. There 
are some similarities between our approach and CommOnCV; 
in fact, both of them: ( i) exploit ontologies to allow companies 
and job seekers to have a common reference for representing 
competencies and tasks; (ii) represent the contents of a job 
proposal or a curriculum vitae by means of suitable tools (i.e., 
XML in our approach, and Semantic Web languages in [37]); 
(Hi) are seeker oriented. As for differences, we may observe 
that: ( i) CommOnCV is centralized and obtrusive whereas our 
approach is mixed and unobtrusive; (/;) CommOnCV is not a 
recommender system. Our system might inherit some features 
from CommOnCV. Specifically, analogously to CommOnCV, 
it might represent a user profile as a network of interests, 
instead of a list of topics; such a choice would allow a 
better comprehension of the relationships existing among user 
interests, as well as a better evaluation of the relevance of 



each interest. However, it should be taken into account that 
the exploitation of a network, instead of a list, for representing 
user interests, would cause an increase of the computational 
complexity of our system. 

G. e-SRS 

In [38] a multi-agent platform, called e-SRS (e-Sales Re- 
cruitment System), for recruiting and benchmarking sales 
persons, is proposed. e-SRS exploits psychometric tools (e.g., 
questionnaires) for classifying each user in a specific category; 
it considers four user categories representing specific behav- 
ioral models (e.g., if a user rarely/frequently trusts others, if a 
user is/is not aggressive, and so on). This categorization can 
be improved by applying a suitable clustering algorithm (fuzzy 
K-means). The obtained results can be considered by a human 
expert for selecting the best candidates. There are some simi- 
larities between our approach and e-SRS. In fact, both of them: 

( i) exploit the agent technology and artificial intelligence tools 
to overcome the limitations of existing recruitment services; 

(ii) construct, handle and exploit accurate user profiles. As 
for differences, we may observe that: (i) e-SRS is company 
oriented; it is obtrusive; (Hi) it is distributed, whereas our 
system is mixed; (iv) it is very specific, since it has been 
conceived for managing the recruitment of sales persons; on 
the contrary, our approach is generic, since it can support 
the recruitment activities in several professional areas; due 
to this reason, e-SRS produces accurate and refined results 
in its application area but its extension to other professional 
categories seems to require a valuable effort. e-SRS might 
be combined with our approach; specifically, in both the 
recruitment and the benchmark of sales persons, it might 
take into account not only the behavioural aspects of a job 
candidate but also his preferences and "constraints". 

Vll. Conclusions and hints for the future 

In this paper we have presented an XML-based, multi- 
agent system for supporting e-recruitment services. We have 
seen that our system is characterized by various interesting 
capabiUties, namely: (i) it allows a uniform management of 
heterogeneous job proposals; as a consequence, a job seeker 
can pose his queries across different, and presumably hetero- 
geneous, job databases; ( ii) it is agent- and XML-based; as a 
consequence, it can easily cooperate with company informa- 
tion systems; the job proposal selection algorithm takes 
into account information about possible interests that the user 
did not consider in the past but that appear to be potentially 
interesting for him in the future; (iv) it is capable of handUng 
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different recommendation strategies; (v) its recommendation 
strategies encompass some mathematic tools already applied 
in many fields. 

In the paper we have illustrated our system in detail, we 
have shown the obtained experimental results and, finally, we 
have compared it with other related systems already presented 
in the literature. 

Various application contexts might benefit from our research 
efforts. Some of the most promising ones are the following: 

• Team Building. In many organizations it is necessary to 
match individuals for composing a team working on the 
same project. Such an activity, often called team building 
or partnership building [39], is becoming an increasingly 
popular strategy for encouraging and boosting production. 
Team building has been studied in a large variety of 
scientific areas: for instance. Human Resource based 
approaches consider the skills and the abilities owned 
by each individual as a leading criterium to compose 
teams; analogously, sociological and psychological ap- 
proaches focus on the role of trust among individuals as 
a key element for the success of a team. Team building 
activities generally consist of two phases, namely: (i) 
identification of potential members of a team, and (ii) 
choice of individuals that really compose a team, among 
the previously defined potential candidates. As observed 
in [39], while various Web sites offer tools for effectively 
carrying out phase (i), decision support for phase (ii) is 
stiU rudimentary. In our opinion, our system might offer 
a concrete contribution in this area. Specifically, since 
our approach manages a large volume of data stored in 
user profiles, it can elaborate this information to compute 
the "affinity degree" existing among two individuals. In 
other words, an analysis of the personal features of an 
individual, as well as of its preferences and exigencies, 
allows managers: (i) to assign each individual to a team; 
(ii) to predict if trust can be established among the 
potential members of a team, even if they do not share a 
common work history. 

• Task Assignment. A classical activity of Human Resource 
managers consists of associating individuals to tasks; this 
activity should be carried out in such a way to exalt the 
skills and the capabilities of individuals. Since this activ- 
ity has a great impact on the business success, an exten- 
sive research on it has been performed in the past [40]. In 
our opinion our system could be used to effectively solve 
the problem of assigning individuals to tasks. Specifically, 
information stored in user profiles could be exploited for 
capturing user aptitudes/preferences whereas the XML- 
based model describing job proposals could be used for 
representing task features. A matching between a user 
profile and a job proposal description could be adopted 
for computing the so called Person-Job Fit (PJF), a pa- 
rameter used by psychologists to measure the suitability 
of a candidate for a specific job. 

• Outsourcing. In recent past businesses have outsourced 
a large variety of activities for improving the quality of 
services and products, for reducing the duration of the 
production cycle and for lowering costs. However, an 



excessive reliance on outsourcing may negatively affect 
business performance: for instance, it might lead to a 
reduced technological innovation [41] or to a limited ca- 
pability of competing with outsourcing partners [42]. Our 
system could be adopted to support business managers to 
decide about the effectiveness of an outsourcing activity; 
specifically, it might analyze the information stored in 
user profiles for assessing the capability of the internal 
components of an organization of executing a given task 
and to evaluate if it is/is not convenient its outsourcing. 
As for future work, we plan to contribute to the imple- 
mentation of the ideas mentioned previously. In addition, 
we plan to enrich our system by providing it with more 
powerful algorithms that try to predict the future interests of 
a user, as well as by designing and implementing modules 
for supporting companies looking for candidates. This last 
feature would allow the realization of a hybrid e-recruitment 
system, embodying the functionalities of both seeker oriented 
and company oriented systems. Finally, we plan to study 
the possibility to integrate our system with an e-learning 
framework for realizing a system capable of suggesting (and, 
next, teaching) to a user the skills he should acquire for 
accessing new, more appealing, job proposals. 
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